Magnifying Lens Abstraction for Stochastic Games with Discounted and Long-run Average Objectives
نویسندگان
چکیده
Turn-based stochastic games and its important subclass Markov decision processes (MDPs) provide models for systems with both probabilistic and nondeterministic behaviors. We consider turnbased stochastic games with two classical quantitative objectives: discounted-sum and long-run average objectives. The game models and the quantitative objectives are widely used in probabilistic verification, planning, optimal inventory control, network protocol and performance analysis. Games and MDPs that model realistic systems often have very large state spaces, and probabilistic abstraction techniques are necessary to handle the state-space explosion. The commonly used full-abstraction techniques do not yield space-savings for systems that have many states with similar value, but does not necessarily have similar transition structure. A semi-abstraction technique, namely Magnifying-lens abstractions (MLA), that clusters states based on value only, disregarding differences in their transition relation was proposed for qualitative objectives (reachability and safety objectives) [8]. In this paper we extend the MLA technique to solve stochastic games with discounted-sum and long-run average objectives. We present the MLA technique based abstraction-refinement algorithm for stochastic games and MDPs with discounted-sum objectives. For long-run average objectives, our solution works for all MDPs and a sub-class of stochastic games where every state has the same value.
منابع مشابه
Strategy Synthesis for Stochastic Games with Multiple Long-Run Objectives
We consider turn-based stochastic games whose winning conditions are conjunctions of satisfaction objectives for long-run average rewards, and address the problem of finding a strategy that almost surely maintains the averages above a given multi-dimensional threshold vector. We show that strategies constructed from Pareto set approximations of expected energy objectives are ε-optimal for the c...
متن کاملStochastic Games: A Tutorial
Game theory [1] is a formalism for the study of competitive interaction in the rich spectrum of relationships ranging between conflict and cooperation. Originally conceived as a mathematical foundation of economics, it proved its robustness by providing new techniques and insights in logic and set theory [15, 13], evolutionary and population biology [22], auction design and implementation, the ...
متن کاملMagnifying-Lens Abstraction for Markov Decision Processes
We present a novel abstraction technique which allows the analysis of reachability and safety properties of Markov decision processes with very large state spaces. The technique, called magnifyinglens abstraction, copes with the state-explosion problem by partitioning the state-space into regions, and by computing upper and lower bounds for reachability and safety properties on the regions, rat...
متن کاملOne-Counter Stochastic Games
We study the computational complexity of basic decision problems for one-counter simple stochastic games (OC-SSGs), under various objectives. OC-SSGs are 2-player turn-based stochastic games played on the transition graph of classic one-counter automata. We study primarily the termination objective, where the goal of one player is to maximize the probability of reaching counter value 0, while t...
متن کاملDiscounted Supermodular Stochastic Games: Theory and Applications
This paper considers a general class of discounted Markov stochastic games characterized by multidimensional state and action spaces with an order structure, and one-period rewards and state transitions satisfying some complementarity and monotonicity conditions. Existence of pure-strategy Markov (Markov-stationary) equilibria for the nite (in nite) horizon game, with nondecreasing and possib...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1107.2132 شماره
صفحات -
تاریخ انتشار 2011